Maneuverless passive range estimation using monocular image sequences

ABSTRACT

A method to estimate range to a moving rigid body from a moving platform using a monocular camera. The method does not require the camera platform to maneuver in order to estimate range. The method relies on identification and tracking of certain principal features of the object. The method extracts a silhouette of an object from an obtained image and identifies two principal linear components of the silhouette. A normalized distance between the point of intersection of the two linear components and a centroid of the silhouette is computed, compared to a data set and used to determine a direction of movement of the object.

DOMESTIC PRIORITY

This application claims the benefit of earlier filed U.S. ProvisionalPatent application No. 61465295, filed Mar. 17, 2011, the entirecontents of which is incorporated herein by reference. This applicationis a continuation-in-part of U.S. patent application No. 13420964, filedMar. 15, 2012.

GOVERNMENT LICENSE RIGHTS

This invention was made with government support under Air Force ContractNo. FA8650-07-C-1200 and DARPA Contract No. W31P4Q-11-C-0118. Thegovernment has certain rights in the invention.

BACKGROUND

Estimating the range to a mobile object using a monocular camera fixedto a moving platform has, in general, remained an unsolved problem forthe past three decades. An effective solution to this problem can beused in autonomous collision detection and avoidance applications forunmanned vehicles, especially unmanned aerial vehicles and surfacevehicles. The technology can also be used to provide situationalawareness information to manned air and water crafts moving in dynamicenvironments. In this disclosure, we present a novel approach to computethe range or distance to a dynamic object using a sequence of monocularimages captured from a moving sensor platform. In what follows, we shalluse the term “intruder” to describe the dynamic object whose range is tobe estimated.

Depending on the sensor resolution, imaging sensors can provide veryaccurate estimates of the relative bearing angle to the intruder. Atlarge distances, an intruder may occupy only a few pixels on an image.The classical approach is to treat the intruder as a point target anduse bearing angle measurements to the tracked point. However, thisapproach is, in general, not sufficient due to the inherentunobservability of the intruder dynamics. We use an example toillustrate this problem.

Let us assume that the sensor platform is initially located at (0,0) ina two-dimensional plane and moves along the positive y-axis withvelocity given by v_(s). Let us assume that the intruder initiallylocated at (x₀, y₀) is moving with velocity (u, v) where u and vrepresent the X and Y components of the velocity. The position of thesensor at any time t is given by (0, v_(s)t). The position of theintruder at time t is given by (x₀+ut, y₀+vt). The bearing angle to theintruder as measured from the sensor at any given time is, therefore,given by arctan (y₀+(v−v_(s))t)/(x₀+ut). Now, let us assume there isanother intruder located initially at (kx₀, ky₀) moving with velocity(ku, kv−(k−1)v_(s)), where k is a positive constant and not equal to 1.Now, notice that the relative bearing to the second intruder measuredfrom the sensor at any time t is given by arctan(ky₀+(kv−(k−1)v_(s))t−v_(s)t)/(kx₀+kut) which is equal to arctan(ky₀+kvt−kv_(s)t)/(kx₀+kut)=arctan (y₀+(v−v_(s))t)/(x₀+ut). Therefore,two intruders with distinct trajectories (k≠1) generate the same bearingangle measurement at the sensor. This clearly illustrates the inherentobservability problem present in bearing-only tracking. Please refer toFIG. 1 for a specific example. In this case, (x₀, y₀)=(500,500), k=2,v_(s)=100, u=−50 and v=50. The sensor position is shown by red circles.The first intruder's position is represented by blue plus signs. Thesecond intruder (position denoted by black asterisks) is moving withvelocity equal to (−50k, 50k−(k−1)100)=(−100,0). The angles θ(0), θ(1)and θ(2) denote the relative bearing to the two intruders at timeinstants 0, 1 and 2. Notice that both intruders subtend the samerelative bearing angles with respect to the sensor, but k≠1 signifyingthat the two intruder trajectories are distinct as can be seen in thefigure.

The conditions of unobservability in a bearing-only tracking problemhave been extensively studied since the late 1970's. This body ofresearch has established that the intruder state is observable, ingeneral, only if the order of sensor dynamics is greater than theintruder dynamics. For an intruder moving with constant velocity, thisimplies that the sensor dynamics must involve an acceleration component.In the event that the sensor dynamics are not of a higher order, thesensor platform must execute a deliberate maneuver involving a higherorder dynamics component to be able to estimate the range. With thegrowing use of Unmanned Aerial Vehicles (UAVs) in recent years, such a“maneuver-based” approach has been proposed as a solution to the passivecamera based Sense and Avoid (SAA) problem: upon detecting an intruder,the UAV maneuvers in order to triangulate and resolve the position ofthe intruder. However, a maneuver-based approach is undesirable in manyways especially in military operations. It may lead to waste of fuel,loss in mission performance, and is in general bad airmanship.

Other research deals with choosing the right coordinate frames andfilters for bearings-only tracking from the point of view of stabilityand unbiasedness. For example, an Extended Kalman Filter (EKF) appliedto bearings-only target tracking has been theoretically analyzed and hasestablished the reasons for the filter bias and instability. A modifiedpolar coordinate system based Kalman Filter has also been proposed toseparate the observable and unobservable dynamics. Further, a bank ofEKFs each with a different initial range estimate referred to as therange-parameterized (RP) tracker has been proposed and shown to performbetter than classical EKF implementations. More recently, particlefilter solutions to bearings-only tracking problem are receivingconsiderable attention and they have been implemented to track bothmaneuvering and non-maneuvering targets. Particle filter and RP EKF havebeen compared and it has been shown that the particle filter is onlymarginally better than the Range Parameterized EKF but is considerablymore robust to errors in the initial target range.

This disclosure presents a novel passive image based ranging approachthat does not require the sensor platform to maneuver. Analyticalresults prove that the approach can be used to estimate the range to anintruder without any sensor maneuver under very general conditions.Tests conducted on real flight-test data have demonstrated thepracticality of the approach.

The invention in this disclosure concerns a novel method to estimate therange to a moving rigid body from a mobile platform using monocularpassive cameras such as electro-optical or infra-red camera mounted onthe platform. The primary application of the proposed technology is inthe area of collision detection and avoidance for unmanned vehicles.Unmanned aerial, surface and ground vehicles equipped with cameras canuse the proposed invention to estimate the range to other moving objectsin their vicinity. The range information can then be used to determinethe trajectory of the moving objects and maintain a safe distance fromthem.

Generally, in a computerized system including a camera mounted on amoving vehicle, wherein the camera acquires consecutively in real time aplurality of images of a moving object within a field of view of thecamera, a method for determining a range of said moving object from saidmoving vehicle comprises the steps of:

(a) detecting the moving object (intruder) within each of said pluralityof images;

(b) identifying two feature points p₁ and p₂ on said detected objectwhere p₁ and p₂ satisfy a certain geometric relationship with thevelocity vector of the detected object including but not limited to thatthe two feature points represent a leading point of the detected objectand a trailing point of the detected object; and

(c) recursively calculating the range to the object based on changes inthe positions of the feature points p₁ and p₂ in the sequential imagesand further based on the assumption that the geometric relationshipmentioned under item (b), including but not limited to the assumptionthat the detected object is traveling in a direction along a lineconnecting feature points p₁ and p₂, is valid

The key aspect of the invention lies in the premise that if you canassume pure translational movement and assume a direction of motionbased on the image, then you can recursively calculate the range withoutthe order of sensor dynamics being greater than the intruder dynamics,i.e. without any extraneous maneuvering.

Accordingly, among the objects of the instant invention are: theprovision of a maneuverless passive ranging method.

Another object of the invention is the provision of a ranging methodwhich extracts feature points on a detected object and provides ageometric relationship between the spatial positions of the featurepoints and the direction of velocity of the object.

Other objects, features and advantages of the invention shall becomeapparent as the description thereof proceeds when considered inconnection with the accompanying illustrative drawings.

DESCRIPTION OF THE DRAWINGS

In the drawings which illustrate the best mode presently contemplatedfor carrying out the present invention:

FIG. 1 is a graph showing the inherent observability problem present inbearing-only tracking;

FIG. 2 is an illustration of a moving sensor platform (unmannedaircraft) and a moving object (intruder) within the field of view of thecamera mounted on the platform;

FIG. 3 is a schematic block diagram of a computer system in accordancewith the invention;

FIG. 4 is an illustration of an image acquired by the camera and anassociated silhouette extracted from the image (nose to tailorientation);

FIG. 5 is another illustration of an image acquired by the camera and anassociated silhouette extracted from the image (wingtip to wingtiporientation);

FIG. 6 is a flowchart of an exemplary software implementation of themethod of the present invention;

FIG. 7 is a flowchart of another exemplary software implementation ofthe method of the present invention;

FIG. 8 is a flowchart of a software implementation of a method ofextracting a silhouette of the image and identifying the pixel locationsi₁, j₁ and i₂, j₂ of feature points p₁, p₂; and

FIG. 9 is a flowchart of a software implementation of another method ofthe present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENT

Referring to FIGS. 2-5, the invention in this disclosure concerns anovel method to estimate a range to a moving rigid body 10 from a mobileplatform 12 using monocular passive cameras 14 such as electro-opticalor infra-red camera mounted on the platform 12. The primary applicationof the proposed technology is in the area of collision detection andavoidance for unmanned vehicles. Unmanned aerial, surface and groundvehicles equipped with cameras can use the proposed invention toestimate the range to other moving objects in their vicinity. Asillustrated in FIG. 2, the exemplary embodiment shows an unmanned aerialvehicle (UAV) 12 (moving platform) estimating range to a civilianaircraft 10 (moving rigid body/intruder). The range information can thenbe used to determine the trajectory of the moving objects and maintain asafe distance from them.

Generally the invention is implemented in a computerized system 16including a CPU 18, memory 20 and a camera mounted on the platform 12.The system 16 communicates with the vehicles navigation system andreceives inputs from the vehicles global positioning system/inertialmeasurement unit (GPS/IMU) 22. The estimated range derived from thesystem 16 can be output back to the navigation system for avoidancemeasures or output to a display 24.

The invention is described in detail herein. In Section 1, there aredescribed some preliminary notions that are used throughout thedocument. In Section 2, there are presented some novel mathematicalresults which form the basis of the invention. In Section 3, themonocular passive ranging (MPR) method is described. In Section 4, thereis described a novel image processing algorithm which can be used as amodule of the MPR algorithm when the rigid body to which the range is tobe estimated is an aircraft. In Section 5, another variant of the MPRmethod is described.

1. Preliminaries

In this section, we describe concepts and notation to be used throughoutthis document. For any time t, let the positions of two feature pointson the intruder be (x₁(t), y₁(t), z₁(t)) and (x₂(t), y₂(t), z₂(t))respectively, where x₁(t), y₁(t), z₁(t), x₂(t), y₂(t), z₂(t)ϵ

, where

is the set of real numbers. Here the coordinates are measured withrespect to a fixed global frame of reference. Let the position of thesensor be (x_(s)(t), y_(s)(t), z_(s)(t)), where again x_(s)(t),y_(s)(t), z_(s)(t) ϵ

. Let r_(i)(t)ϵ

³ be the position vector of feature point i with respect to the sensorat time t, where iϵ{1,2}. In other words, we have,r _(i)(t):=[x _(i)(t)−x _(s)(t) y _(i)(t)−y _(s)(t) z _(i)(t)−z_(s)(t)].Given a vector Xϵ

^(N), let ∥X∥ denote the Euclidean norm of the vector. We let X(t)^((n))denote the nth derivative of X(t) with respect to time, assuming thatthe nth derivative exists. For notational convenience, we also assumethat X(t)⁽⁰⁾ is equivalent to X(t). We will also use ∥X(t)∥ as analternate representation of ∥X(t)∥⁽¹⁾. Similarly, we let ∥X(t)∥^((n))denote the nth derivative of ∥X(t)∥ with respect to time. In whatfollows, for simplicity we shall omit the dependence on time wherever itis clear from the context.

2. Novel Mathematical Conditions for Maneuverless Monocular PassiveRanging

In this section, we present the results on the observability of theintruder states. Intuitively speaking, an intruder state is said to beobservable if the state can estimated from the available measurements.In particular, we are interested in observing the range to the intruderat a given time.

Lemma 1. Let the intruder be a rigid body undergoing a puretranslational motion. Let (x₁(t), y₁(t), z₁(t)) and (x₂(t), y₂(t),z₂(t)) be the locations of two points on the intruder at time t.Assuming that the motion is sufficiently smooth for the first orderderivatives to exist, we have {dot over (x)}₁={dot over (x)}₂; {dot over(y)}₁={dot over (y)}₂; ż₁=ż₂.

The proof is a straightforward consequence of the fact that the body isrigid and undergoes no rotation. We are now ready to state our mainresults on observability. The results are summarized by the followingtheorems.

Theorem 1. Let (x(t), y(t), z(t)) represent the trajectory of a givenpoint on a mobile rigid intruder at time t. Likewise, let (x_(s)(t),y_(s)(t), z_(s)(t)) represent the trajectory of the sensor. Let r(t)denote the vector (x(t)−x_(s)(t), y(t)−y_(s)(t), z(t)−z_(s)(t)). Let themeasurement vector be defined by h(t):=r(t)/∥r(t)∥ and let h_(x)(t),h_(y)(t) and h_(z)(t) denote the three components of h(t). Let thefollowing conditions be true:

-   (i) there exists a positive integer N≥2 such that r(t)^((i)) exists    for all t and for all iϵ{1, . . . ,2N−2};-   (ii) r(t)^((N)) is equal to zero.-   (iii) h_(x) ^((i)), h_(y) ^((i)) and h_(z) ^((i)) are non zero for    all i ϵ {0, . . . ,N−2}.

Then the quantity ∥r(t)∥⁽¹⁾/∥r(t)∥ is observable if the followingconditions are true at time t.

-   (i)

${\frac{h_{x}^{(1)}}{h_{y}^{(1)}} \neq \frac{h_{x}}{h_{y}}},$and

-   (ii) for N>2 there does not exist a set of real numbers {a₁, . . . ,    a_(N−2)} with at least one of them not equal to zero such that both    the following conditions are true:

$\begin{matrix}{{h_{x}^{({N - 1})} = {\sum\limits_{i = 1}^{N - 2}{h_{x}^{({N - 1 - i})}\alpha_{i}}}},} & (1) \\{h_{y}^{({N - 1})} = {\sum\limits_{i = 1}^{N - 2}{h_{x}^{({N - 1 - i})}{\alpha_{i}.}}}} & (2)\end{matrix}$

Theorem 2. Let (x₁(t), y₁(t), z₁(t)) and (x₂(t), y₂(t), z₂(t)) be thelocations of two points on a mobile rigid intruder at time t. Let therigid body's motion be purely translational. Let [u_(s)(t) v_(s)(t)w_(s)(t)] and [u(t) v(t) w(t)] be the velocity vectors of the sensor andthe intruder at time t. Let RϵSO(3) be a rotation matrix such that(1/∥[u(t) v(t) w(t)]∥)[u(t) v(t) w(t)]^(T)=(1/∥[x₁(t)−x₂(t) y₁(t)−y₂(t)z₁(t)−z₂(t)]∥)R[x₁(t)−x₂(t) y₁(t)−y₂(t) z₁(t)−z₂(t)]^(T). Let r_(i)(t)denote the vector(x_(i)(t)−x_(s)(t),y_(i)(t)−y_(s)(t),z_(i)(t)−z_(s)(t)) for iϵ{1,2}. Letthe measurements be given by h_(i)(t):=r_(i)(t)/∥r_(i)t∥ for iϵ{1,2}.Let h_(i) _(x) , h_(i) _(y) and h_(i) _(z) denote the x, y and zcomponents of h_(i). Suppose that the following conditions hold:

-   (i)

$\frac{{r_{1}}^{(1)}}{r_{1}}$is observable;

-   (ii) {dot over (β)}₂≠0 and y₂≠π/2, where β_(i)=arctan

$\frac{h_{i_{x}}}{h_{i_{y}}}$and

${\gamma_{i} = {\arctan\;\frac{h_{i_{z}}}{\sqrt{\left( {h_{i_{x}}^{2} + h_{i_{y}}^{2}} \right)}}}};$

-   (iii) The rotation matrix R is known; and-   (iv) The velocity of the sensor is non-zero.

Then, the states of the intruder are observable as long as the vectors[u_(s)(t) v_(s)(t) w_(s)(t)] and [u(t) v(t) w(t)] are not parallel.

Theorem 3. Let the intruder be a rigid body undergoing a translationmotion. Let (x₁(t), y₁(t), z₁(t)) and (x₂(t), y₂(t), z₂(t)) be thelocations of two feature points on the intruder at time t. Let [u_(s)(t)v_(s)(t) w_(s)(t)] and [u(t) v(t) w(t)] be the velocity vector of thesensor and the intruder at time t. Let α ϵ [0, π/2] be the angle betweenthe vectors [u(t) v(t) w(t)] and [x₁(t)−x₂(t) y₁(t)−y₂(t) z₁(t)−z₂(t)].For parallel vectors, we assume that α=0 and for perpendicular vectorsα=π/2.

Suppose that the following conditions hold:

-   (i)

$\frac{{r_{1}}^{(1)}}{r_{1}}$is observable;

-   (ii) {dot over (β)}₂≠0, and y₂≠π/2;-   (iii) The angle α is known; and-   (iv) The velocity of the sensor is non-zero.

Then, the following statements hold true.

-   (i) If α=0 and (u_(s), v_(s), w_(s)) is not parallel to the vector    (x₁−x₂, y₁−y₂, z₁−z₂), then the intruder states are observable;-   (ii) If α=π/2 and the vector (u_(s), v_(s), w_(s)) is not    perpendicular to the vector (x₁−x₂, y₁−y₂, z₁−z₂), then the intruder    states are observable;

One of the critical assumptions in the above theorem is that (u_(s),v_(s), w_(s)) is not parallel or perpendicular to the vector(x₁−x₂,y₁−y₂, z₁−z₂). This again corresponds to events that are highlyimprobable Now that we have established the mathematical conditions forthe range to an intruder to be observable, we present a novel monocularpassive ranging method that does not require the sensor platform tomaneuver.

3. A Novel Maneuverless Monocular Passive Ranging (MPR) Method

In this section, a novel MPR method based on the applications of Theorem1 and Theorems 2 and 3 is described. We assume that there is a mobileplatform containing a passive monocular camera system as the sensor. Thesensor system is used to detect the presence of the other moving objects10 in the vicinity and estimate the range to the objects. (See FIGS. 4and 5) A number of algorithms for detection of objects in images havebeen proposed in the prior art and we assume that a suitable algorithmis available for use with our proposed invention—our invention concernsthe estimation of range to a moving object (intruder) once it has beendetected. The following steps describe our method assuming thatdetection has been accomplished and that there is no noise in any of themeasurements.

-   (i) Get the pixel locations (i₁, j₁) and (i₂, j₂) of two feature    points in the image of the intruder, denoted for short by p₁ and p₂,    respectively, such that either (1) the rotation matrix R is known    or (2) the angle α is known and is equal to either 0 or π/2 radians    The matrix R is defined in Theorem 2 (See FIG. 6—Case 1). The angle    α as defined in Theorem 3 is equal to 0 or π/2 (See FIG. 7—Case 2).-   (ii) Assume a suitable value of N, where N is a positive integer    such that r₁ ^((N))=0 is a good approximation of the dynamics of the    spatial location of feature point p₁ on the intruder.-   (iv) Measure the angles β₁, β₂, y₁ and y₂ where the quantities are    defined in Theorem 2. This can be accomplished by knowing the    intrinsic parameters of the camera (focal length, field of view and    the camera mounting angle), the extrinsic parameters of the camera    (yaw, pitch and roll angles of the sensor platform obtained from the    onboard IMU) and the locations p₁ and p₂ in the image plane. This is    equivalent to the measurement of h_(x) ₁ , h_(y) ₁ , h_(z) ₁ , h_(x)    ₂ , h_(y) ₂ and h_(z) ₂ where h_(x) _(i) , h_(y) _(i) and h_(z) _(i)    are the x, y and z components of r_(i)/∥r_(i)∥.-   (v) Estimate the first 2N−3 derivatives of h_(x) _(i) , h_(y) _(i)    and h_(z) _(i) using the current value of h_(x) _(i) , h_(y) _(i)    and h_(z) _(i) and their past values stored in the memory 20 Use the    constructive proof of Theorem 1 to estimate ∥r₁∥⁽¹⁾/∥r₁∥. Notice    that N≥2 because N=1 implies that the range is constant.-   (vi) Now we know ∥r₁∥⁽¹⁾/∥r₁∥ and the first derivatives of β₁, β₂,    y₁ and y₂. Depending on whether we know R or whether we know that α    is 0 or π/2 radians, use the method in the proofs of Theorem 2 and    Theorem 3 respectively to estimate ∥r₁∥.

For ease of reference of the reader, the equations of dynamics andmeasurements for a Recursive Filter Implementation for Cases 1 and 2 arelaid out below.

Recursive Filter Implementation for Section 3—Case (1)

Dynamics Equationsx ₁ ^((N)) −x _(s) ^((N))=0, y ₁ ^((N)) −y _(s) ^((N))=0, z ₁ ^((N)) −z_(s) ^((N))=0x ₂ ^((N)) −x _(s) ^((N))=0, y ₂ ^((N)) −y _(s) ^((N))=0, z ₂ ^((N)) −z_(s) ^((N))=0x ₁ ^((i)) =x ₂ ^((i)), 1≤i≤Ny ₁ ^((i)) =y ₂ ^((i)), 1≤i≤Nz ₁ ^((i)) =z ₂ ^((i)), 1≤i≤Nu(R ₂₁(x ₂ −x ₁)+R ₂₂(y ₂ −y ₁)+R ₂₃(z ₂ z ₁)) =v(R ₁₁(x ₂ −x ₁)+R ₁₂(y₂y ₁)+R ₁₃(z ₂ −z ₁))u(R ₃₁(x ₂ −x ₁)+R ₃₂(y ₂ y ₁)+R ₃₃(z ₂ −z ₁)) =w(R ₁₁(x ₂ −x ₁)+R ₁₂(y₂ y ₁)+R ₁₃(z ₂ −z ₁))Here N is a suitable positive integer, the superscript (N) denotes theNth derivative with respect to time, u=x₁ ⁽¹⁾, v=y₁ ⁽¹⁾, w=z₁ ⁽¹⁾ andR_(ij) is the ij element of the rotation matrix R, and (x_(s), y_(s),z_(s)) is the 3D position of the sensor platform.Measurement Equations

$\beta_{1} = {\arctan\;\frac{h_{1_{x}}}{h_{1_{y}}}}$$\beta_{2} = {\arctan\;\frac{h_{2_{x}}}{h_{2_{y}}}}$$\gamma_{1} = {\arctan\;\frac{h_{1_{z}}}{\sqrt{\left( {h_{1_{x}}^{2} + h_{1_{y}}^{2}} \right)}}}$$\gamma_{2} = {\arctan\;\frac{h_{2_{z}}}{\sqrt{\left( {h_{2_{x}}^{2} + h_{2_{y}}^{2}} \right)}}}$where$h_{1_{x}} = \frac{x_{1} - x_{s}}{\sqrt{\left( {x_{1} - x_{s}} \right)^{2} + \left( {y_{1} - y_{s}} \right)^{2} + \left( {z_{1} - z_{s}} \right)^{2}}}$$h_{1_{y}} = \frac{y_{1} - y_{s}}{\sqrt{\left( {x_{1} - x_{s}} \right)^{2} + \left( {y_{1} - y_{s}} \right)^{2} + \left( {z_{1} - z_{s}} \right)^{2}}}$$h_{1_{z}} = \frac{z_{1} - z_{s}}{\sqrt{\left( {x_{1} - x_{s}} \right)^{2} + \left( {y_{1} - y_{s}} \right)^{2} + \left( {z_{1} - z_{s}} \right)^{2}}}$$h_{2_{x}} = \frac{x_{2} - x_{s}}{\sqrt{\left( {x_{2} - x_{s}} \right)^{2} + \left( {y_{2} - y_{s}} \right)^{2} + \left( {z_{2} - z_{s}} \right)^{2}}}$$h_{2_{y}} = \frac{y_{2} - y_{s}}{\sqrt{\left( {x_{2} - x_{s}} \right)^{2} + \left( {y_{2} - y_{s}} \right)^{2} + \left( {z_{2} - z_{s}} \right)^{2}}}$$h_{2_{z}} = \frac{z_{2} - z_{s}}{\sqrt{\left( {x_{2} - x_{s}} \right)^{2} + \left( {y_{2} - y_{s}} \right)^{2} + \left( {z_{2} - z_{s}} \right)^{2}}}$Knowing the dynamics and the measurement equations, standard non-linearfiltering techniques such as the Extended Kalman Filter can beimplemented to get an estimate of the states (x₁, y₁, z₁), (x₂, y₂, z₂),and (u, v, w). Let the estimates be denoted by ({circumflex over (x)}₁,ŷ₁, {circumflex over (z)}₁), ({circumflex over (x)}₂, ŷ₂, {circumflexover (z)}₂), and (û, {circumflex over (v)}, ŵ). Once these states areestimated, the estimated range is given as follows:

$r^{est} = \sqrt{\left( {\frac{{\hat{x}}_{1} + {\hat{x}}_{2}}{2} - x_{s}} \right)^{2} + \left( {\frac{{\hat{y}}_{1} + {\hat{y}}_{2}}{2} - y_{s}} \right)^{2} + \left( {\frac{{\hat{z}}_{1} + {\hat{z}}_{2}}{2} - z_{s}} \right)^{2}}$$V^{est} = \sqrt{{\hat{u}}^{2} + {\hat{v}}^{2} + {\hat{w}}^{2}}$

Recursive Filter Implementation for Section 4.3—Case (2) and for Section5 (below)

Dynamics Equationsx ₁ ^((N)) −x _(s) ^((N))=0, y ₁ ^((N)) −y _(s) ^((N))=0, z ₁ ^((N)) −z_(s) ^((N))=0x ₂ ^((N)) −x _(s) ^((N))=0, y ₂ ^((N)) −y _(s) ^((N))=0, z ₂ ^((N)) −z_(s) ^((N))=0x ₁ ^((i)) =x ₂ ^((i)), 1≤i≤Ny ₁ ^((i)) =y ₂ ^((i)), 1≤i≤Nz ₁ ^((i)) =z ₂ ^((i)), 1≤i≤NIf α=0 radians, then:v(x ₂ −x ₁)=u(y ₂ −y ₁)w(y ₂ −y ₁)=v(z ₂ −z ₁)If α=π/2 radians, then:u(x ₂ −x ₁)+v(y ₂ −y ₁)+w(z ₂ −z ₁)=0.Here N is a suitable positive integer, the superscript (N) denotes theNth derivative with respect to time, u=x₁ ⁽¹⁾, v=y₁ ⁽¹⁾, w=z₁ ⁽¹⁾ and(x_(s), y_(s), z_(s)) is the 3D position of the sensor platform.Measurement Equations

$\beta_{1} = {\arctan\;\frac{h_{1_{x}}}{h_{1_{y}}}}$$\beta_{2} = {\arctan\;\frac{h_{2_{x}}}{h_{2_{y}}}}$$\gamma_{1} = {\arctan\;\frac{h_{1_{z}}}{\sqrt{\left( {h_{1_{x}}^{2} + h_{1_{y}}^{2}} \right)}}}$$\gamma_{2} = {\arctan\;\frac{h_{2_{z}}}{\sqrt{\left( {h_{2_{x}}^{2} + h_{2_{y}}^{2}} \right)}}}$where$h_{1_{x}} = \frac{x_{1} - x_{s}}{\sqrt{\left( {x_{1} - x_{s}} \right)^{2} + \left( {y_{1} - y_{s}} \right)^{2} + \left( {z_{1} - z_{s}} \right)^{2}}}$$h_{1_{y}} = \frac{y_{1} - y_{s}}{\sqrt{\left( {x_{1} - x_{s}} \right)^{2} + \left( {y_{1} - y_{s}} \right)^{2} + \left( {z_{1} - z_{s}} \right)^{2}}}$$h_{1_{z}} = \frac{z_{1} - z_{s}}{\sqrt{\left( {x_{1} - x_{s}} \right)^{2} + \left( {y_{1} - y_{s}} \right)^{2} + \left( {z_{1} - z_{s}} \right)^{2}}}$$h_{2_{x}} = \frac{x_{2} - x_{s}}{\sqrt{\left( {x_{2} - x_{s}} \right)^{2} + \left( {y_{2} - y_{s}} \right)^{2} + \left( {z_{2} - z_{s}} \right)^{2}}}$$h_{2_{y}} = \frac{y_{2} - y_{s}}{\sqrt{\left( {x_{2} - x_{s}} \right)^{2} + \left( {y_{2} - y_{s}} \right)^{2} + \left( {z_{2} - z_{s}} \right)^{2}}}$$h_{2_{z}} = \frac{z_{2} - z_{s}}{\sqrt{\left( {x_{2} - x_{s}} \right)^{2} + \left( {y_{2} - y_{s}} \right)^{2} + \left( {z_{2} - z_{s}} \right)^{2}}}$Knowing the dynamics and the measurement equations, standard non-linearfiltering techniques such as the Extended Kalman Filter can beimplemented to get an estimate of the states (x₁, y₁, z₁), (x₂, y₂, z₂),and (u, v, w). Let the estimates be denoted by ({circumflex over (x)}₁,ŷ₁, {circumflex over (z)}₁), ({circumflex over (x)}₂, ŷ₂, {circumflexover (z)}₂), and (û, {circumflex over (v)}, ŵ). Once these states areestimated, the estimated range is given as follows.If α=0 radians, then:

$r_{\alpha = 0}^{est} = \sqrt{\left( {\frac{{\hat{x}}_{1} + {\hat{x}}_{2}}{2} - x_{s}} \right)^{2} + \left( {\frac{{\hat{y}}_{1} + {\hat{y}}_{2}}{2} - y_{s}} \right)^{2} + \left( {\frac{{\hat{z}}_{1} + {\hat{z}}_{2}}{2} - z_{s}} \right)^{2}}$$V_{\alpha = 0}^{est} = \sqrt{{\hat{u}}^{2} + {\hat{v}}^{2} + {\hat{w}}^{2}}$If α=π/2 radians, then:

$r_{\alpha = {\pi/2}}^{est} = \sqrt{\left( {\frac{{\hat{x}}_{1} + {\hat{x}}_{2}}{2} - x_{s}} \right)^{2} + \left( {\frac{{\hat{y}}_{1} + {\hat{y}}_{2}}{2} - y_{s}} \right)^{2} + \left( {\frac{{\hat{z}}_{1} + {\hat{z}}_{2}}{2} - z_{s}} \right)^{2}}$$V_{\alpha = {\pi/2}}^{est} = \sqrt{{\hat{u}}^{2} + {\hat{v}}^{2} + {\hat{w}}^{2}}$

In a practical situation, measurements are always corrupted by noise. Insuch cases, a recursive filter implementation, such as the ExtendedKalman Filter, can be used to estimate the range and other states of theintruder.

An exemplary recursive EKF method is described below

Let

$\alpha_{correlation} = \frac{\left( {{\left( {x_{1} - x_{2}} \right)u} + {\left( {y_{1} - y_{2}} \right)v} + {\left( {z_{1} - z_{2}} \right)w}} \right)}{{{\left( {x_{1} - x_{2}} \right),\left( {y_{1} - y_{2}} \right),\left( {z_{1} - z_{2}} \right)}}{{u,v,w}}}$

Where the states x₁, y₁, z₁, x₂, y₂, z₂, u, v, w are recursivelyestimated without the knowledge of whether α=0 or απ/2

We determine thatα=π/2 if α_(correlation)≤δα=0 if α_(correlation)>1−δ

where δ is a tuning parameter

Furthermore, instead of using a single EKF, one could use a bank ofEKF's, or a particle filter for recursively computing the range. In thiscase, one can compute α=0 or α=π/2 for each particle in the bank andthen use a voting mechanism to determine if α=0 or α=π/2. If the numberof particles with α=0 is larger than the number of particles for whichα=π/2, then we make a determination that α=0 and vice versa.

4. Image Processing for Feature Point Identification in Aircraft Images

In the previous section, there is described a monocular passive rangingmethod. In step (i) of the method, two feature points on the intruderare identified such that the the rotation matrix R is known or that theangle α is known and is equal to either 0 or π/2 radians. In thissection, a novel method to identify two feature points such that theangle α is equal to either 0 or π/2 radians is described. The methodalso provides an algorithm to check if the angle α is equal to 0 or π/2radians. This method is applicable to the case when the intruder is anaircraft. A software implementation is laid out in FIG. 8. The basicsteps involved are as follows.

-   (i) Extraction of the intruder aircraft silhouette. FIGS. 4 and 5    demonstrate the silhouettes extracted by our algorithm on two sample    images of commercial airliners downloaded from the internet. The    left images show the original images whereas the right images show    the respective silhouettes.-   (ii) Extraction of the two principal components of the silhouette.    In order to extract the principal components, a line searching    method similar to Hough transform is used. For this method, the    binary aircraft silhouette image is used. First the straight line L₁    which contains the maximum number of silhouette pixels is estimated.    This serves as the longer principal component. We also estimate a    second principal component. This is defined as the straight line L₂    that contains the maximum number of silhouette pixels subject to the    condition that it makes an angle of greater than 45 degree with the    longer principal component. This approach has been observed to yield    better results than more traditional principal component analyses    approaches such as Singular Value Decomposition.

In FIGS. 4 and 5 (right image), the two principal components areindicated by the lines L₁ and L₂ on the silhouette. The end pointsL_(1a), L_(1b) of the longer primary principal component correspond toeither the end points of the fuselage (nose and tail) or to the wingtips. For example, in FIG. 4 (top right), the end points of the longerprincipal component line correspond to the head and the tail of theaircraft, whereas the end points L_(1a), L_(1b) of the longer principalcomponent in FIG. 5 correspond to the two wing tips of the aircraft.

-   (iii) Computation of the normalized distance between the point of    intersection of the two principal components and the centroid of the    silhouette. For example, in FIGS. 4, the point of intersection of    the two principal components is indicated by an asterisk, and the    centroid of the silhouette is indicated by a circle. The normalized    distance is defined as the distance between the asterisk and the    circle normalized by the length of the longer principal component    which in this case is the length of line L₁ in the silhouette. The    normalized distance can be used to check whether the end points    correspond to the end points of the fuselage or to the wing tips or    neither. The intuition is that if the end points are on the fuselage    then the normalized distance should be larger than if the end points    are the wing tips.-   (iv) Creation of a training data set from images for which the    identity of the end points of the longer principal component is    known. For this data set, the normalized distances are computed. We    downloaded 50 aircraft images from the internet—25 corresponding to    the case where the end points are on the fuselage and the remaining    25 corresponding to the case where the end points are the wing tips.    The normalized distances are extracted for each of the images in the    training data set. It is observed that these distances can be    clustered into two intervals, [0,0.3] and [0.5,1]. The former and    the latter intervals correspond to the wing tips and the fuselage    images respectively. This is in accordance with our intuition.-   (v) Comparison of the extracted normalized distances in the test    images with the normalized distances in the training data. For    robust classification, instead of using a single test image we use a    sequence of consecutive images, compute the normalized distances for    each of the images, and then perform statistical T-tests to test the    hypotheses of whether the mean of the normalized distances in the    sequence of the test images is greater than 0.5 or less than 0.3. If    the hypothesis that the mean is greater than 0.5 is true, then this    implies that for the images in the sequence the end points    correspond to the fuselage or equivalently that α is equal to zero    radians. If the other hypothesis is true then it implies that the    end points correspond to the wing tips or that α is equal to π/2    radians.

4a. Silhouette Extraction by Image Segmentation

The exemplary method below can be used as an alternative to the methodset forth in FIG. 8 and paragraphs 93-98 described above.

A silhouette extraction module is used to identify the two featurepoints of the object in the field of view. This works by firstperforming image segmentation in a neighborhood of the detected aircraftto obtain the aircraft silhouette, and then identifying two featurepoints from the silhouette.

Segmentation:

The image segmentation module receives a sequence of image windows of afixed size around the detected aircraft. The steps in the imagesegmentation process are:

1. Compute image features: These are extracted by

-   -   Performing Principal Components Analysis (PCA) on a small        subsampled image.    -   Using the PCA to compute one or two color components with        maximum variance from the input color image    -   Optionally, using the gradient of the first component.    -   Pixel coordinates are also used as features to ensure a        connected silhouette.    -   Normalizing all features and weight them in a manner that        improves the resulting segmentation.

2. Shape prior: The segmentation resulting from the previous frame isalso used as a feature to maintain continuity of the silhouette. Theshape prior is registered with the new image by shifting the shape priortill the centroids of the edge maps of the previous and current framematch. Optionally, the shift may be estimated by matching the brightestor darkest pixels in the current and previous frames.

3. Segmentation: A fast DBSCAN algorithm is used to segment the featureimage into multiple clusters. The largest clusters and those touchingthe image boundary are marked as background and the rest are marked asforeground. The foreground cluster is the silhouette of the intruderaircraft.

Robust Feature Point identification:

The feature points of the silhouette are further identified in a mannerrobust to errors in silhouette extraction. A straight line is fit to thesilhouette using least squares fitting. This is the principal axis ofthe silhouette. All silhouette points are projected to the principalaxis and a truncated histogram of the points is calculated along theprincipal axis. The centroid and interquartile range (IQR) arecalculated from the histogram. The robust feature point estimates arethe two points a distance of IQR away from the centroid along theprincipal axis.

5. Likelihood Testing Based Range Estimation

In this section, another method for estimating the range to the intruderis described. The difference of this method from the method in Section 3Case 2 (See FIG. 6 —Case 2) is that we only need to know that the angleα is either 0 or π/2 radians but it is not required to know which one ofthe two it is. A software implementation is laid out in FIG. 9. Thesensor system is used to detect the presence of the other moving objects10 in the vicinity and estimate the range to the objects. (See FIGS. 4and 5) A number of algorithms for detection of objects in images havebeen proposed in the prior art and we assume that a suitable algorithmis available for use with our proposed invention—our invention concernsthe estimation of range to a moving object (intruder) once it has beendetected. The following steps describe our method assuming thatdetection has been accomplished and that there is no noise in any of themeasurements.

-   (i) Get the pixel locations (i₁, j₁) and (i₂, j₂) of two feature    points in the image of the intruder, denoted for short by p₁ and p₂,    respectively, such that the angle α is equal to either 0 or π/2    radians but it is not known which one of the two it is. If the    intruder is an aircraft, steps (i) and (ii) of the method in Section    4 can be used to find the end points of the longer principal    component of the silhouette of the intruder in the image. These end    points serve as points p₁ and p₂.-   (ii) Assuming the angle α is equal to zero, estimate the range to    the intruder by application of the method described in Section 3.    Once the range to the intruder is estimated, other states such as    the range rate can also be estimated since ∥r(t)∥⁽¹⁾/ ∥r(t)∥ is    observable according to Theorem 1. Using the knowledge of the    bearing angles, and bearing angle derivatives, estimate the velocity    of the intruder. Let the estimated range and magnitude of the    velocity of the intruder be denoted by r_(α=0) ^(est) and V_(α=0)    ^(est).-   (iii) Assuming the angle α is equal to π/2, estimate the range and    range rate of the intruder. Using the knowledge of the bearing    angles, and bearing angle derivatives, estimate the velocity of the    intruder. Let the estimated range and magnitude of the velocity of    the intruder be denoted by r_(α=π/2) ^(est) and V_(α=π/2) ^(est).-   (iv) Let p_(range)(r_(α=0) ^(est)) be the likelihood of the range    estimate being equal to r_(α=0) ^(est) based on prior sensor    detection ranges. For example, if r_(α=0) ^(est) is equal to 10    nautical miles and the sensor detection range can never be more than    5 nautical miles, then the likelihood p_(range)(r_(α=0) ^(est))    should be small. Likewise, if r_(α=0) ^(est) is a small value then    too the likelihood p_(range)(r_(α=0) ^(est)) should be small because    then the intruder should have been detected earlier. Likewise, let    p_(range)(r_(α=π/2) ^(est)) be the likelihood of the range estimate    being equal to r_(α=π/2) ^(est). Similarly, one can estimate    likelihoods p_(speed)(V_(α=0) ^(est)) and p_(speed)(V_(α=π/2)    ^(est)) based on typical speeds of the intruders in the domain of    operation.-   (v) If

$\frac{{p_{range}\left( r_{\alpha = 0}^{est} \right)}{p_{speed}\left( V_{\alpha = 0}^{est} \right)}}{{p_{range}\left( r_{\alpha = {\pi/2}}^{est} \right)}{p_{speed}\left( V_{\alpha = {\pi/2}}^{est} \right)}}$is greater than a certain threshold M then α is equal to 0 and thecorrect range estimate is r_(α=0) ^(est).

If, on the other hand,

$\frac{{p_{range}\left( r_{\alpha = 0}^{est} \right)}{p_{speed}\left( V_{\alpha = 0}^{est} \right)}}{{p_{range}\left( r_{\alpha = {\pi/2}}^{est} \right)}{p_{speed}\left( V_{\alpha = {\pi/2}}^{est} \right)}}$is less than a certain threshold m then α is equal to π/2 and thecorrect range estimate is r_(α=π/2) ^(est).

The previous method requires the knowledge of the likelihood functions.Any suitable likelihood function can be used as long as it satisfied twoproperties. First, it should be a positive real valued function. Second,the function must assume a strictly greater value for an argument thatis more likely to occur than another argument that is less likely tooccur. The particular choice of the likelihood function depends on theneeds of the user and domain of operation. For example, for use in anunmanned aircraft collision avoidance application, the likelihoodfunctions should be designed after a careful study of the types ofintruder aircraft operating in the airspace and their performancecapabilities as well as the type of passive sensor being used. Likewise,the thresholds M and m are tunable design parameters and should bechosen based on the application scenario and system objectives.

6. Collision Detection

Previously we established that one can compute r¹(t) and r²(t)—the rangeto the wto feature points using the method described in the patent. Wenow present a method to predict an imminent collision.

We introduce the following notation. Let t_(current) be the time atwhich the range to the feature points is computed. Define constantsT_(safe) and R_(safe) as safe time and safe distance which allow for theeither the ownship or the intruder to maneuver and avoid an impendingcollision. These constants depend on the dynamics of the ownship and theintruder. They are defined by the needs of the application. We determinethat a collision is imminent if at a critical time t*≥t_(current) in thefuture if |r^(i)(t*)|≤R_(safe) for i=1 or 2, t*−t_(current)≥0 andt*−t_(current)≤T_(safe)

In general case one can determine t* as

$\underset{t}{argmin}{{r^{i}(t)}}$for i=1 or 2. Depending on the motion of the intruder and ownship t*might not be unique. We check for the collision detection condition fort* that is greater than t_(current) and closest to it among all thepossible t*.

For the special case where intruder and the ownship are not acceleratingi.e. both of them have linear motion then t* is computed using theformula

${t^{*} = {{- {\underset{i}{argmax}\left\lbrack \frac{\left( {x_{i} - x_{s}} \right) + \left( {y_{i} - y_{s}} \right) + \left( {z_{i} - z_{s}} \right)}{\left( {u - u_{s}} \right) + \left( {v - v_{s}} \right) + \left( {w - w_{s}} \right)} \right\rbrack}} + t_{current}}},{i = 1},2$

It can therefore be seen that the present invention provides amaneuverless passive ranging method which is based on extracting featurepoints on a detected object and assuming a direction of movement basedon those selected feature points. For these reasons, the instantinvention is believed to represent a significant advancement in the artwhich has substantial commercial merit.

While there is shown and described herein certain specific structureembodying the invention, it will be manifest to those skilled in the artthat various modifications and rearrangements of the parts may be madewithout departing from the spirit and scope of the underlying inventiveconcept and that the same is not limited to the particular forms hereinshown and described except insofar as indicated by the scope of theappended claims.

What is claimed is:
 1. In a computerized system including a cameramounted on a moving vehicle, a method of predicting a collision betweena foreign moving object and said moving vehicle, the method comprisingthe steps of: acquiring with said camera, consecutively in real time, aplurality of images of said foreign moving object; detecting saidforeign moving object within each of said plurality of images;identifying two related feature points p₁ and p₂ on said detectedforeign moving object within each of said plurality of images;calculating ranges r¹(t) and r²(t) to said identified feature points p₁and p₂ on said detected foreign moving object based on changes in thepositions of the feature points p₁ and p₂ in the sequential images; anddetermining that a collision condition is imminent if at a timet*≥t_(current) in the future if |r^(i)(t*)|≤R_(safe) for i=1 or 2,t*−t_(current)≥0 and t*−t_(current)≤T_(safe) where t_(current) is thetime at which the ranges r¹(t) and r²(t) to the feature points p₁ and p₂are computed and where T_(safe) and R_(safe) define a safe time and asafe distance which allow for the moving vehicle or the foreign movingobject to maneuver to avoid a collision.
 2. The method of claim 1wherein $t^{*} = {\underset{t}{argmin}{{r^{i}(t)}}}$ for i=1 or 2 fora general case.
 3. The method of claim 1 further comprising the step ofchecking for said collision condition for t* that is greater thant_(current) and closest to it among all possible t*.
 4. The method ofclaim 2 further comprising the step of checking for said collisioncondition for t* that is greater than t_(current) and closest to itamong all possible t*.
 5. The method of claim 1 where${t^{*} = {{- {\underset{i}{argmax}\left\lbrack \frac{\left( {x_{i} - x_{s}} \right) + \left( {y_{i} - y_{s}} \right) + \left( {z_{i} - z_{s}} \right)}{\left( {u - u_{s}} \right) + \left( {v - v_{s}} \right) + \left( {w - w_{s}} \right)} \right\rbrack}} + t_{current}}},{i = 1},2$for special cases where the foreign moving object and the moving vehicleare not accelerating.
 6. The method of claim 2 where${t^{*} = {{- {\underset{i}{argmax}\left\lbrack \frac{\left( {x_{i} - x_{s}} \right) + \left( {y_{i} - y_{s}} \right) + \left( {z_{i} - z_{s}} \right)}{\left( {u - u_{s}} \right) + \left( {v - v_{s}} \right) + \left( {w - w_{s}} \right)} \right\rbrack}} + t_{current}}},{i = 1},2$for special cases where the foreign moving object and the moving vehicleare not accelerating.
 7. The method of claim 6 further comprising thestep of checking for said collision condition for t* that is greaterthan t_(current) and closest to it among all possible t*.
 8. The methodof claim 1 wherein said collision condition is selectively used by saidcomputerized system to automatically alter a trajectory of said movingvehicle.
 9. The method of claim 6 wherein said collision condition isselectively used by said computerized system to automatically alter atrajectory of said moving vehicle.
 10. In a computerized systemincluding a camera mounted on a moving vehicle, a method of predicting acollision between a foreign moving object and said moving vehicle, themethod comprising the steps of acquiring with said camera, consecutivelyin real time, a plurality of images of said foreign moving object;detecting said foreign moving object within each of said plurality ofimages; identifying two feature points p₁ and p₂ on said detectedforeign moving object within each of said plurality of images;calculating ranges r¹(t) and r²(t) to said identified feature points p₁and p₂ on said detected foreign moving object based on changes in thepositions of the feature points p₁ and p₂ in the sequential images; anddetermining that a collision condition is imminent if at a timet*≥t_(current) in the future if |r^(i)(t*)|≤R_(safe) for i=1 or 2,t*−t_(current)≥0 and t*−t_(current)≤T_(safe) where t_(current) is thetime at which the ranges r¹(t) and r²(t) to the feature points p₁ and p₂are computed and where T_(safe) and R_(safe) define a safe time and asafe distance which allow for the moving vehicle or the foreign movingobject to maneuver to avoid a collision; wherein: the two feature pointsp₁ and p₂ on said detected foreign moving object satisfy a predeterminedgeometric relationship with a velocity vector of said detected foreignmoving object, said step of identifying said feature points p₁ and p₂further comprising the step of determining pixel locations (i₁, j₁) and(i₂, j₂) of said feature points p₁ and p₂ within said image; and thecalculating ranges step further includes recursively calculating therange of said detected foreign moving object based on changes in thepositions of the feature points p₁ and p₂ in the sequential images andfurther based on said predetermined geometric relationship, said step ofrecursively calculating said range further comprising the step ofconverting said pixel locations to relative azimuth and elevation anglesdenoted by β₁, β₂, y₁, y₂, wherein the step of recursively calculatingsaid range comprises recursively calculating said range where an angle αis known and is equal to either 0 or π/2 radians, and the followingconditions are defined: (x₁, y₁, z₁) and (x₂, y₂, z₂) represent 3Dlocations of said feature points (i₁, j₁) and (i₂, j₂); [u, v, w]represents a 3D velocity vector of the object; angle α is the anglebetween the vectors [u v w] and [x₁−x₂ y₁−y₂ z₁−z₂]; and a determinationof whether α is equal to either 0 or π/2 is made using an ExtendedKalman Filter (EKF) where$\alpha_{correlation} = \frac{\left( {{\left( {x_{1} - x_{2}} \right)u} + {\left( {y_{1} - y_{2}} \right)v} + {\left( {z_{1} - z_{2}} \right)w}} \right)}{{{\left( {x_{1} - x_{2}} \right),\left( {y_{1} - y_{2}} \right),\left( {z_{1} - z_{2}} \right)}}{{u,v,w}}}$where the states x₁, y₁, z₁, x₂, y₂, z₂, u, v, w are recursivelyestimated without the knowledge of whether α=0 or α=π/2; andα=π/2 if α_(correlation)≥δα=0 if α_(correlation)<1−δ where δ is a tuning parameter.
 11. The methodof claim 10 wherein α is estimated using a bank of EKFs, and α=0 orα=π/2 is computed for each filter in each bank and then a votingmechanism is used to determine if α=0 or απ/2 where if the number offilters with α=0 is larger than the number of filters for which α=π/2,then α=0, and if the number of filters with α=π/2 is larger than thenumber of filters for which α=0, then α=π/2.
 12. The method of claim 10wherein α is estimated using a particle filter, and α=0 or α=π/2 iscomputed for each particle in the filter and then a voting mechanism isused to determine if α=0 or α=π/2 where if the number of particles withα=0 is larger than the number of particles for which α=π/2, then α=0,and if the number of particles with α=π/2 is larger than the number ofparticles for which α=0, then α=π/2.
 13. In a computerized systemincluding a camera mounted on a moving vehicle, a method of predicting acollision between a foreign moving object and said moving vehicle, themethod comprising the steps of: acquiring with said camera,consecutively in real time, a plurality of images of said foreign movingobject; detecting said foreign moving object within each of saidplurality of images; identifying two feature points p₁ and p₂ on saiddetected foreign moving object within each of said plurality of images;calculating ranges r¹(t) and r²(t) to said identified feature points p₁and p₂ on said detected foreign moving object based on changes in thepositions of the feature points p₁ and p₂ in the sequential images; anddetermining that a collision condition is imminent if at a timet*≥t_(current) in the future if |r^(i)(t*)|≤R_(safe) for i=1 or 2,t*−t_(current)≥0 and t*−t_(current)≤T_(safe) where t_(current) is thetime at which the ranges r¹(t) and r²(t) to the feature points p₁ and p₂are computed and where T_(safe) and R_(safe) define a safe time and asafe distance which allow for the moving vehicle or the foreign movingobject to maneuver to avoid a collision; wherein the two feature pointsp₁ and p₂ on said detected foreign moving object satisfy a predeterminedgeometric relationship with a velocity vector of said detected foreignmoving object, said step of identifying said feature points p₁ and p₂further comprising the step of determining pixel locations (i₁, j₁) and(i₂, j₂) of said feature points p₁ and p₂ within said image; and whereinthe calculating the ranges step further includes recursively calculatingthe range of said detected foreign moving object based on changes in thepositions of the feature points p₁ and p₂ in the sequential images andfurther based on said predetermined geometric relationship, wherein thestep of recursively calculating said range comprises recursivelycalculating said range where an angle α is known and is equal to either0 or π/2 radians, and the following conditions are defined: (x₁, y₁, z₁)and (x₂, y₂, z₂) represent 3D locations of said feature points (i₁, j₁)and (i₂, j₂); [u, v, w] represents a 3D velocity vector of the object;angle α is the angle between the vectors [u v w] and [x₁−x₂ y₁−y₂z₁−z₂]; said step of determining pixel locations (i₁, j₁) and (i₂, j₂)comprises extracting a silhouette of said moving object by imagesegmentation.
 14. The method of claim 13 wherein said step of imagesegmentation comprises the steps of: receiving a sequence of images offixed size around said detected moving object; computing image featuresfrom said images by extracting principal components from said images andperforming principal component analysis (PCA) on a sub-sample of images;using the PCA to compute at least one color component with a maximumvariance from an input color image; normalizing said image features; andweighting the image features to improve segmentation; segmenting theimage features into a plurality of clusters; identifying the largestclusters; marking the largest clusters and those clusters which touch animage boundary as background; marking the remaining clusters asforeground and identifying said remaining clusters as said silhouette ofsaid moving object.
 15. The method of claim 14 wherein a segmentationresult from a previous image of said sequence of images is used tomaintain continuity of said silhouette between images.
 16. The method ofclaim 15, wherein said silhouette from said previous image (shape prior)is registered with a current image by shifting the shape prior until acentroid of an edge map of the previous image matches the current image.17. The method of claim 16 wherein said step of shifting is estimated bymatching a brightest or darkest pixel in the current image and theprevious image.
 18. In a computerized system including a camera mountedon a moving vehicle, a method of predicting a collision between aforeign moving object and said moving vehicle, the method comprising thesteps of: acquiring with said camera, consecutively in real time, aplurality of images of said foreign moving object; detecting saidforeign moving object within each of said plurality of images;identifying two feature points p₁ and p₂ on said detected foreign movingobject within each of said plurality of images; calculating ranges r¹(t)and r²(t) to said identified feature points p₁ and p₂ on said detectedforeign moving object based on changes in the positions of the featurepoints p₁ and p₂ in the sequential images; and determining that acollision condition is imminent if at a time t*≥t_(current) in thefuture if |r^(i)(t*)|≤R_(safe) for i=1 or 2, t*−t_(current)≥0 andt*−t_(current)≤T_(safe) where t_(current) is the time at which theranges r¹(t) and r²(t) to the feature points p₁ and p₂ are computed andwhere T_(safe) and R_(safe) define a safe time and a safe distance whichallow for the moving vehicle or the foreign moving object to maneuver toavoid a collision; wherein the two feature points p₁ and p₂ are tworobust feature points p₁ and p₂ on said detected foreign moving object,where p₁ and p₂ satisfy a predetermined geometric relationship with avelocity vector of said detected foreign moving object, said step ofidentifying said feature points p₁ and p₂ further comprising the step ofdetermining pixel locations (i₁, j₁) and (i₂, j₂) of said robust featurepoints p₁ and p₂ within said image; and wherein the calculating theranges step further includes recursively calculating the range of saidobject based on changes in the positions of the robust feature points p₁and p₂ in the sequential images and further based on said predeterminedgeometric relationship, said step of recursively calculating said rangefurther comprising the step of converting said pixel locations torelative azimuth and elevation angles denoted by β₁, β₂, y₁, y₂, andwherein said step of determining pixel locations (i₁, j₁) and (i₂, j₂)of said robust feature points p₁ and p₂ within said image comprises thesteps of extracting a silhouette of said moving object comprising aplurality of pixels associated with said detected foreign moving object,fitting a straight line to the silhouette using a least squares methodwherein said straight line defines a principal axis of said silhouette,projecting all silhouette points to the principal axis, calculating atruncated histogram of said silhouette points, and calculating acentroid and interquartile range (IQR) from said histogram, wherein saidrobust feature points p₁ and p₂ comprise two points having a distance ofIQR away from the centroid along the principal axis.
 19. The method ofclaim 18 wherein the step of recursively calculating said range furthercomprises recursively calculating said range where an angle α is knownand is equal to either 0 or π/2 radians, and the following conditionsare defined: (x₁, y₁, z₁) and (x₂, y₂, z₂) represent 3D locations ofsaid feature points (i₁, j₁) and (i₂, j₂); [u, v, w] represents a 3Dvelocity vector of the object; and angle α is the angle between thevectors [u v w] and [x₁−x₂ y₁−y₂ z₁−z₂].